1. Field of the Invention
This invention relates to vision substitution systems, specifically to a system method and apparatus for presenting audio and tactile representations of visual items.
2. Description of Related Art
Devices have previously been invented that substitute for aspects of vision with another sense, particularly hearing and touch, and can be useful to blind and partially sighted people, and to those temporarily unable to see, for example aircraft pilots blinded by smoke. Fournier d'Albe's 1912 Exploring Optophone highlighted the edges of objects through tone sounds, and his Reading Optophone (described in U.S. Pat. No. 1,350,954) presented the shapes of letters by scanning along lines of type with a column of five spots of light, the reflected light from each spot controlling the volume of a different musical note, producing a characteristic series of chords for each letter. His book “The Moon Element” states “Incidental uses of the optophone consist in the examination of pictures, photographs, maps, and dress materials”. The invention described in U.S. Pat. No. 5,097,326 and “The vOICe” vision substitution system present live images via sound. U.S. Pat. No. 6,963,656 describes using musical sequences to convey features of images. Similar approaches have been used to “sonify” the lines on a two-dimensional line graph. Typically a left-to-right column scan is performed, height is mapped to pitch, and intensity to volume (either dark- or light-sounding).
In the tactile modality, several inventions have allowed visual features to be presented via touch, usually via an array of tactile actuators (typically vibro-tactile or electro-tactile) acting on parts of the user's body, for example Telesensory's™ Optacon™ finger-read vibro-tactile array; U.S. Pat. No. 6,430,450 and Wicab's™ Brainport™ tongue-placed electro-tactile display; U.S. Pat. Appl. No. 2010/0151426 and EyePlusPlus's™ Forehead Sensory Recognition System electro-tactile display; electromechanical tactile abdomen displays; and KGS's™ 8 by 8/64-dot tactile display. The present applicant's U.S. Pat. No. 8,239,032 describes a palm-read tactile display. Furthermore, standard programmable braille displays 34 FIG. 2 can be configured to act as displays of simple visual representations.
The present applicant's U.S. Pat. No. 8,239,032 entitled “Audiotactile Vision Substitution System” discloses a system for presenting one or several apparently-moving speech sounds and corresponding tactile effects (referred to as “tracers”) that follow the paths of the shapes of lineal features present in visual representations (i.e. “trace out the shapes”), at the same time as presenting information related to the shapes, with distinct audiotactile indicator effects highlighting corners within the shapes. The present applicant's published UK Pat. Appl. GB1101732.4 entitled “Improved Audiotactile Vision System” discloses using buzzing “tracers” (known as “buzz tracks”) to improve the perception of shapes. The specification and drawings of U.S. Pat. No. 8,239,032 and UK Pat. Appl. No. GB1101732.4 are incorporated by reference, and copies are obtainable from the Internet and elsewhere. Parts of the present applicant's previous inventions are described in this specification.
The methods described in the present applicant's earlier patents are effective for presenting items that can be summarized via lineal effects (for example via outlines of items 140 FIG. 9, or via a “symbolic tracer paths” 146). To convey the two-dimensional arrangement of the content of an item the system used “Layouts” 42 FIGS. 5 & 180 FIG. 13, which categorically presented the locations of an item's material, via speech sounds 43, braille 44, or coded Morse-like taps 45. (“Items” can be for example objects within a visual representation; regular regions of a visual representation; abstract shapes; etc.) Whereas the earlier inventions described tactile methods used to convey the layout of items via tactile displays for example via braille (or via the “Optacon” or other tactile arrays), in the audio modality the layout was conveyed via coded speech sounds that semantically described the arrangement of matter. Additionally, sets of apparently-moving multiple tracer speech or non-speech “poly/racers” FIGS. 14 to 18 could produce optophone-like effects that helped convey the nature and extent of the material within an item—a set of tracers, arranged approximately in a line, move approximately perpendicular to the line, to “sweep out” item areas, in a similar manner to an optophone. (For solid “blob” shapes the extent is to some degree also conveyed via the locations presented via an “outline” or “medial” tracer.)